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AN IMAGE RECOGNITION-PROCESSING DEVICE AND METHOD 
USING PATTERN ELEMENTS RESIDING ON A RECORDING MEDIUM 

BACKGROU ND OF THE INVENTION 

5 I. FIELD OF THE INVENTION 

This invention generally relates to a device and method to prevent the 

counterfeiting of paper money, stock certificates, or similar documents. More 

specifically, it relates to an image recognition device which is capable of recognizing a 

specific input image even though the input image has been enlarged or reduced in size. 

10 

= = II. DESCRIPTION OF THE RELATED ART 

~ In recent years, the capabilities of full-color copy machines have become so 

- ' sophisticated that it is often impossible to distinguish a copy from the original image 

' ° with the naked eye. At the same time, such copying machines have become readily 

= 15 available. Thus, there is an increased danger that copiers will be misused to counterfeit 
3 documents, which may not legally be reproduced, such as banknotes and stock 

certificates. Various devices have been developed to prevent such counterfeiting. 



Some of these devices pre-record a specific image of a banknote or other 
20 document, which may not be legally reproduced. Then, these devices determine 
whether the pre-recorded specific image exists in an input image which someone is 
attempting to copy. The method used to determine whether the image is recognized 
(i.e., pattern matching) entails evaluating the input image's distance from the template, 
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or the relative correlation between a target pattern (the template) and die input image 
the device is attempting to recognize (the object of recognition). 

If the device does recognize the target pattern in the input image, it employs any 
of various means to prevent copying, such as covering the paper entirely with black ink 
or halting the process. 

Size is an essential element used in the technique described above. The size 
evaluation consists of determining if the input image and target pattern are exactly the 
same size. The input image and the target pattern must be the same size for the device 
to prevent copying. If the shapes in the two images are the same (i.e., the figures are 
similar) but their dimensions are different, the device will not recognize the input image 
as identical to the target pattern. As a result, if someone were to use the copier's zoom 
function to reduce or enlarge the input image to be copied, the size of the input image 
data, which the image recognition device received from the copier, would not match the 
target pattern stored in the device, and the device would conclude that the input image 
was not one for which copying should be prohibited. 

A digital color copier normally reads the image loaded on the glass by moving an 
image sensor with one line of CCD (in the scanning direction) at a given speed. If the 
input image has been reduced or enlarged, the speed of the image sensor can be 
increased or decreased respectively. This is necessary because the quantity of data in the 
feed direction, which is received per unit of time, will differ with magnification. 
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It is understood that humans recognize shapes by a complex process. However, 
human are not as skilled in evaluating size. In fact, recognizing (or discriminating) 
small differences in size is difficult for most people. An item which is successfully copied 
because the image recognition device does not recognize the input image as the same 
size as the reference image may nevertheless be indistinguishable to the human eye. 

If the magnification (either enlargement or reduction) is ten percent or more, 
anyone can easily recognize a copy by comparing it directly with the original bill or 
document and seeing that it is the wrong size; but seeing only the copy makes it much 
more difficult. Furthermore, if it is a bill which a person seldom sees, foreign currency 
for example, the task is even harder. 

To address this problem, the image processing device disclosed in Japanese 
Patent Publication 6-237379 was developed. During preprocessing, this invention 
extracts a specified portion (which varies with the magnification) of the input image data 
based on the magnification it receives from the copier. This process makes the 
resolution of the extracted image data fixed at any magnification or zoom level. In this 
way the same processing can be executed in every subsequent unit without regard to the 
magnification, so that the likelihood of recognition will not vary with magnification. 

The prior art device described above is effective for machines such as copiers 
which can transmit magnification or zoom data to it, but it cannot be used for machines 
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which are unable to provide accurate magnification data. For example, an image 
processing system which consists of a scanner, a personal computer and a printer, inputs 
processes and reproduces the image by itself. This makes it difficult to obtain the sort of 
reliable index of magnification which a copier can readily provide. Thus, the 
5 recognition processing employing magnification disclosed in the Japanese Patent 
Publication 6-237379 cannot be applied to this sort of image processing system. 

The rapid development of electronic technology has improved the resolution of 
all the devices discussed above and enabled accurate full color printing. For this reason 
1=0 we must have a reliable way to halt the reproduction of images containing specified 
= target patterns. 



,7 SUMMARY OF THE INVENTION 

Z The present invention provides an image recognition device, a recording 

1=5 medium, and an image processing device which are capable of executing recognition 
processing accurately and evaluating the input image even when the input image data 
and target pattern are different sizes due to magnification or reduction. 



The preferred embodiment of this invention has a pattern element recognizing 
20 means to recognize the pattern elements obtained by dividing an input image into a 

plurality of regions which are compared with corresponding target pattern elements, and 
a pattern detection means to detect the relative position of the pattern elements, which 
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determines whether the pattern elements detected by the pattern element recognizing 
means are arranged properly with respect to each other. 

In another embodiment of the present invention, the device has a dictionary 
generating unit which stores dictionary data for each pattern element in the reference 
image; an element matching unit, which compares and matches the input image pattern 
' data against the target pattern data stored in the dictionary unit; a storage unit for 
arrangement data (corresponding to "arrangement data generating unit 4"), which 
stores data representing the arrangement of the pattern elements; and a pattern 
detection unit, which based on the output of the element matching unit, determines 
whether the target pattern can be found in the input image data. 

The image processing device of this invention executes specified image 
recognition processing on the input image data. After recognizing a match between the 
target pattern and the input image data being subject to the processing, by one of die 
above methods, this device prevents the normal image processing from occurring. In 
the preferred embodiment, this image processing device is realized by a personal 
computer 12. Any device other than a personal computer 12 which executes various 
image processing routines would also be considered an image processing device for the 
purpose of this invention, naturally including a copy machine. If used with an image 
processing device, such as a copier, which is capable of providing a magnification value, 
the device may employ the magnification value to establish the arrangement data which 
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will serve as the basis for comparison, or it may establish the arrangement data without 
resorting to the magnification data from the copier. 

The recording medium of this invention contains an image recognition program 
5 which sends commands to the computer to execute processing to match the input 
image data received against die dictionary data for each element in the pattern and 
processing to determine whether the arrangement of the pattern elements detected by 
the matching indicates the existence of the pattern. 

The target pattern (i.e., dictionary data) used to recognize the specified image 
consists of pattern elements obtained by dividing the pattern into a number of regions. 
The specified image is recognized by matching the input image data against the 
dictionary data. By executing image recognition independentiy for a number of regions 
consisting of a small quantity of data (i.e., the pattern elements), we can minimize the 
effect of the discrepancy which shows up in each region due to the change in scale when 
the input image has been magnified or reduced. 

If recognition is attempted using an entire pattern whose scale has been changed, 
the dimensional discrepancy (i.e., the discrepancy due to the number of elements) will 
20 be large, and the pattern will not be recognized. However, if recognition is executed on 
discrete pattern elements, the effect of the change in scale will be reduced, and the 
pattern can be recognized and detected. 
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Using smaller regions to mitigate the effects of magnification or reduction will 
cause the recognition rate to drop because false positives will be detected in images 
whose pattern elements could have been detected correctiy regardless of scale; such false 
positives can be avoided by tying recognition to arrangement data. This will minimize 
5 the drop in recognition rate due to a slight change in scale. 

TittTPF nKSCRIPTTONT OF THE DRAWINGS 

The foregoing and other advantages and features of the invention will become 
more apparent from the detailed description of preferred embodiments of the invention 

10 given below with reference to the accompanying drawings in which: 

: Figure 1 is a block diagram of an image recognition device according to a 

! preferred embodiment of the present invention; 

11 Figure 2 shows an example of a target pattern (reference pattern) to be 
. I recognized; 



Figure 3 (a) shows an example of how the specified reference image might be 
divided into pattern elements (dictionary data); 



20 



Figure 3 (b) shows several examples of arrangement data; 
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Figure 4 shows an example of an input pattern to be compared with the target 
pattern; 

Figure 5 (a) illustrates a target pattern used for image recognition in Figure 5(b); 

5 

Figure 5 (b) illustrates the difficulty experienced in the prior art, where the input 
image data is larger than the target (reference) pattern; 

Figure 6 illustrates image recognition in accordance with a preferred 
10 embodiment of the present invention; 

Figure 7 shows another example of a target pattern to be recognized; 

Figure 8 shows how the pattern in Figure 7 might be divided into pattern 
elements; 

Figure 9 illustrates how recognition processing could be accomplished using the 
pattern elements shown in Figure 8; 

20 Figure 10 shows a preferred embodiment of an image processing system 

containing the image processing device of the present invention; 
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Figure 1 1 shows the configuration of a system in which the recording medium of 
the present invention is implemented; and 

Figure 12 is a block diagram of the configuration of a system in which the 
5 recording medium of the present invention is implemented. 

DETAILED DESCRIPTION OF PREFERRED EMBODIMENTS 

Referring now to the drawings, where like reference numerals designate like 
elements, there is shown in Figure 1 a device for image recognition using pattern 
10 elements. The device of Figure 1 comprises a dictionary generating unit 1 which stores 
dictionary data concerning the target patterns to be detected. This dictionary data is 
obtained by dividing the target pattern (reference pattern) into a number of regions and 
storing each as a pattern element. 

15 For example, according to a prior art algorithm, the target pattern 5 shown in 

Figure 2 would be recorded in the dictionary generating unit 1 as a target pattern in its 
entirety. With the present invention, the target pattern is divided into a number of 
(here, six) regions, and dictionary data for each region is stored as pattern elements 5a 
through 5f. 

20 

Using the dictionary data stored in dictionary generating unit 1 , element 
matching unit 2 executes matching processing on the input image. It sends the results 
of the matching to the next stage, pattern detection unit 3. For the dictionary data 
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associated with the target pattern in Figure 2, the six pattern elements 5a through 5f are 
read out (i.e., taken out of dictionary generating unit 1), and matching is executed with 
respect to each element. A determination is made as to whether the target pattern 
elements can be found in each input image. If they are found, the matching results are 
sent, together with their position data, to pattern detection unit 3. In this case the data 
consist of a set indicating which pattern element was found in which location. 

In this embodiment, the location of the element consists of the coordinates of 
the center of the detected pattern element. In other words, element matching unit 2 
also has the capacity to detect this center location. Since the processing by which the 
input image data is matched against the individual target pattern elements can be the 
same as that used in prior art devices, we will not provide a detailed explanation of it 
here. 

Based on the matching results it receives from element matching unit 2 (the data 
as to the target pattern elements which match each of the input image pattern elements 
and the position data) and the arrangement data concerning target pattern elements 5a 
through 5f which it receives from arrangement data generating unit 4, pattern detection 
unit 3 determines whether the pattern elements in the input image, which match all of 
the target pattern elements, are in the proper locations. If they are, it concludes that the 
input image data that was received contains the target pattern, and it outputs the result 
that the target pattern has been recognized. 
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As can be seen in Figure 3 (a), the arrangement data stored in arrangement data 
generating unit 4 consists of the relative position coordinates which are defined by the 
center positions a through f of pattern elements 5a through 5f. The pattern shown in 
Figure 3 (a) is identical to pattern 5, which is to say that it represents the arrangement 
5 data for a magnification of 100%. However, the arrangement data in this embodiment 
is not limited to 100% only. Data for other magnifications are stored as well. As can be 
seen in Figure 3(b), arrangement data for a magnification of 115% consists of the 
relative position coordinates of center points a' through f , and arrangement data for 
magnification of 95% consists of the relative position coordinates of center points a" 
3.0 through P. 

I Arrangement data generating unit 4 outputs a magnification which was 

previously estimated or inferred from the input image data along with the arrangement 
data for those elements. The magnification is estimated from the magnification of the 

=15 object to be recognized. In actuality, a number of magnifications are provided as 
output sequentially. To infer the magnification based on the input image data, the 
technique disclosed in Japanese Patent Publication 9-132763 may be applied. If an 
accurate magnification value can be obtained from the exterior, as in the case of a 
copier, arrangement data appropriate for the given magnification is provided as output. 

20 

We shall next give a more detailed explanation of this invention while discussing 
the actual recognition processing. If original pattern or target pattern 5 has the 
dimensions and shape shown in Figure 2, let us assume that there is an input pattern 6 
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to be recognized, as shown in Figure 4. Input pattern 6 has been magnified to 115% of 
the size of the original target pattern 5. With this small difference, there is a real 
possibility that a person viewing the input image pattern could mistakenly think it is 
identical to target pattern 5. 

If a prior art image recognition method is used, the entire target pattern 5 in 
Figure 2 will be used as a reference. When the target pattern 5, shown by dotted lines 
in Figure 5 (a), is matched against the actual input pattern 6, the size difference will 
cause a significant discrepancy (see Figure 5 (b)), the probability of receiving a positive 
image recognition (the matching value) will be low, and target pattern 5 will not be 
recognized in the input pattern 6. 

With the current embodiment of the present invention, as can be seen in Figure 
6, each of the six target pattern elements 5a through 5f is compared to the 
corresponding input pattern element individually so that each is treated as a small 
independent region. The discrepancies caused by the magnification or reduction of the 
input image will for the most part be limited within a given input pattern element 
because the comparison is conducted in each region of the pattern elements. This 
regional comparison can result a sufficient accuracy in the matching process. 

For example, in the example shown in Figure 6, element matching unit 2 will 
detect all of input pattern elements 6a through 6f and extract the center coordinates 
(indicated by white circles in the drawing) for each element. It will then transmit these 
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coordinate values along with a descriptive notation (a number) for each element to 
pattern detection unit 3 as the result of the recognition routine. 

Pattern detection unit 3 compares the coordinate values sequentially with the 
plurality of reference arrangement data associated with each magnification which it 
receives from arrangement data generating unit 4. From this comparison, it determines 
whether the relative positional relationship of the input pattern elements that were 
detected matches the arrangement data for any of the magnifications, and so determines 
whether the input and target patterns match. In the example shown in Figure 6, the 
relationship matches the arrangement data for a magnification of 115%, so it is 
determined that the detected input pattern matches the target pattern. A result 
indicating this match is output from pattern detection unit 3. 

Resolution is one of the crucial elements in determining the presence of a pattern 
element. For example, when each comparison region is reduced for each input pattern 
element according to this embodiment, but elements 5a through 5f are changed to the 
higher resolution, then the detection can not be precisely carried out. This is because 
the total number of pixels in an image will increase or decrease according to the 
magnification ratio. 

When one region of the image in target pattern 5 contains a square with 100 
pixels in it, the input pattern magnified to 115% will have 115 pixels. The magnified 
input pattern of 115% will increase by 15 pixels. When that same target pattern 5 is 
divided into ten equal segments, each segment will contain a square consisting of 10 
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pixels. So even though the entire input pattern has been magnified to 115% of its 
original size, each input pattern element will consist of only 11.5 pixels, an increase of 
only one to two pixels. The input pattern elements can thus be detected in the target 
pattern elements almost without any effect from the magnification, thus ensuring that 
both matching and detection can be accomplished with acceptable accuracy. 

However, if the target pattern of the actual dimensions is divided into ten equal 
regions and the resolution is increased by, for example, a factor of ten, each segment 
forming a pattern element will be a square containing 100 pixels. In this case, if the 
entire image is magnified to 115% of its original size, there will be fifteen extra pixels in 
each segment, and the accuracy of detection will surfer. 

For this reason, the resolution of the pattern elements should be low enough 
such that no effect is experienced from a slight magnification. The discrepancy due to 
magnification should be kept within the range of accidental error in the input image. 
For our purposes, "slight magnification" refers to a low magnification which an ordinary 
person cannot discriminate or recognize with his naked eye. Magnification ratios large 
enough to distinguish with the naked eye need not be addressed. If the dimensions of 
the copy are obviously different from those of the original, so that a counterfeit 
banknote can easily be distinguished at a glance, there is no need for countermeasures. 

With this sort of low resolution, the recognition rate with respect to individual 
pattern elements will not be high enough. In other word, patterns which are not 
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pattern elements will mistakenly be recognized. (This is why we cannot lower the 
resolution of the entire pattern as a whole and match it directly against the entire 
pattern). This drop in the recognition rate can be offset by recognizing a number of 
regions (i.e., pattern elements) at the same time and by using arrangement data 
concerning their relative arrangement to aid in the recognition. 

Figure 7 shows another example of a pattern to be recognized. The pattern 5 
shown in Figure 2 consisted of a number of shapes, so it could easily be divided into 
elements; however, the invention is not limited to this sort of pattern only, but is also 
applicable for patterns like that shown in Figure 7, which do not lend themselves easily 
to being divided. 

The pattern shown in Figure 7 is continuous, so it would be difficult to divide it 
along the shapes in the pattern. It could, however, be divided using the mesh pattern in 
Figure 8 into twenty-five pattern elements 7a. The black circle in the center of each 
element 7a marks its center position. If, as is shown in Figure 9, this pattern is enlarged, 
pattern elements 7a can all be moved slighdy away from each other. The corresponding 
portions of pattern 9 can then be matched against the pattern elements, and the pattern 
will be recognized despite the magnification. The image recognition device for this 
purpose would be configured as in Figure 1. 

The pattern may be divided into elements using schemes other than a mesh. The 
pattern elements could, for example, be circular, which would make the effect of a 
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change in scale isotropic. However, if die pattern is divided into circular pattern 
elements which are not overlapped, there will be gaps between the circles. To eliminate 
these gaps, we can create more elements to overlap the circles and cover these gaps. 
When overlapping elements are used, the accuracy of matching elements may be 
affected; however, this is an effective way to deal with a magnified or reduced image, 
since the overall accuracy will be enhanced. Hexagonal elements, which might be 
viewed as intermediate between round and square elements, could also be used to create 
a pattern like a honeycomb. In addition to those mentioned, any number of other 
shapes could also be used. 

Figure 10 illustrates a preferred embodiment of an image processing system 
comprising an image processing device in accordance with the present invention. As 
illustrated in Figure 10, in the image processing system the document is read by a 
scanner 10, the image input device, and the resulting image data is sent to personal 
computer 12, the image processing device. Personal computer 12 executes the 
prescribed processing on the input image data it receives and outputs the processed data 
to printer 14, the image generating device. Using the data it receives, printer 14 prints 
an image on the paper and outputs it. 

The processing executed by personal computer 12 might include changing the 
scale of the image, modifying it, correcting its color, or any of a variety of possible 
operations. The image could also be stored in any of a variety of recording media 
instead of being sent directly to printer 14. The processing to enlarge or reduce the 
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image need not be limited to personal computer 12. It could also be executed when 
scanner 10 reads the image or just before printer 14 outputs it. The image input device 
need not be a scanner. It could also be a digital camera (either still or video) or any of a 
variety of devices. The scheme used to input the image need not be limited to the use 
of an image input device like scanner 10, as described above. A recording medium such 
as floppy disc or magnetic optical disc could also be used, or the image could be 
inputted electronically. 

In this embodiment, image recognition device 16 is connected to personal 
computer 12. Image recognition device 16 has the configuration shown in Figure 1. It 
receives the input image data from personal computer 12 and executes the specified 
image recognition routine. If it determines that the image data it is processing contains 
the target image, it sends personal computer 12 a message indicating that result. When 
it receives such a message, personal computer 12 interrupts its normal image processing. 
It may simply stop processing the image, or it may reduce the resolution, cause the 
entire image to be a given color (for example, black), or superimpose an alphanumeric 
character (i.e., letter) or mark on the image so that anyone can see at a glance that the 
bill or document is counterfeit or a copy. 

As is shown by the dotted lines in the Figure 10, the result of recognition may be 
output either to personal computer 12 or to printer 14. Also, personal computer 12 is 



not 



the only possible source of the input image data. The input image data may also 
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originate in scanner 10 or printer 14. In case the input image is received from scanner 
10, the result of recognition may be sent directly to scanner 10. 

Personal computer 12 may have application software to execute the actual image 
processing. Image recognition device 16 may be a discrete device (including a board) 
which is attached to personal computer 12, or it may consist of application software 
installed in personal computer 12. In this case, image recognition device 16 would be 
built into the personal computer 12 shown in Figure 10. In an image processing 
system, image recognition device 16 could be installed in scanner 10 or printer 14 at the 
user's discretion. 

In the discussion above, the arrangement data needed to deal with changes in the 
scale of the image is stored ahead of time in arrangement data generating unit 4 of 
Figure 1. However, the invention is not limited to this scheme only. Alternatively, 
arrangement data for 100% of original size could be stored, and data for different 
magnifications could be generated by enlarging or reducing the image as needed for 
each image presented. The processing to accomplish this magnification or reduction 
could be also executed by pattern detection unit 3. 

In the embodiment discussed above, the program system was stored ahead of 
time in the personal computer 12, which was treated as a single device. However, the 
invention is not limited to this case only. There could be an application program which 
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caused the computer to execute all the processing routines, and this program could be 
stored in a given storage medium. 

For example, as is shown in Figures 11 and 12, floppy disk 20 or CD-ROM 21 
could be used as the storage medium. The program stored on storage medium 20 or 
21 would then be installed in hard drive unit 25, which is connected to (or installed in) 
personal computer 24 via floppy drive 22 or CD-ROM drive 23. In addition, personal 
computer 24 will acquire the configuration discussed in the above embodiment. 

In this embodiment of Figure 12, the matching processing executed in element 
matching unit 2 and pattern detection unit 3 and the program which detects the center 
positions and determines the matching of the arrangement data could actually be 
installed in hard drive unit 25. Then when needed, this functionality could be 
transmitted from hard drive unit 25 to internal memory 26, where processing units 2 
and 3 would be constructed. In this way all the processing could be executed at a high 
speed. The dictionary data in generating unit 1 and the arrangement data in generating 
unit 4 could also be stored in hard drive unit 25 or internal memory 26. 

If only a basic set of arrangement data is stored in arrangement data generating 
unit 4 and data is generated anew for each magnified or reduced image, this generating 
function may be performed by a program which is stored, like processing units 2 and 3 
discussed above, on hard drive unit 25. When needed, this program would be 
transmitted from hard drive unit 25 to internal memory 26. 
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As is discussed above, the image recognition device, recording medium and 
image processing device according to this invention can recognize an image which has 
been distorted so that its size and/or shape differ from the original image. This is 
performed by dividing the image into smaller elements and comparing each input image 
element with the corresponding element of a target pattern. This approach absorbs the 
differences in size and shape and thus allows the distorted image to be recognized. In 
addition to the data for each element of the image, data concerning the relative 
arrangement of the elements is also considered in the final determination of whether the 
input image data contains the target image. This use of relative position data effectively 
minimizes any detrimental effect the scheme might have on recognition rate. 



While certain embodiments of the invention have been described and illustrated 
above, the invention is not limited to these specific embodiments as numerous 
modifications, changes and substitutions of equivalent elements can be made without 
departing form the spirit and scope of the invention. Accordingly, the scope of the 
present invention is not to be considered as limited by the specifics of the particular 
structures which have been described and illustrated, but is only limited by the scope ol 
the appended claims. 

What is claimed as new and desired to be protected by Letters Patent of the 
United States is: 



1096245 vl; NHV901I.DOC 



